InFi: End-to-End Learning to Filter Input for Resource-Efficiency in Mobile-Centric Inference

Mobile-centric AI applications have high requirements for the resource-efficiency of model inference. Input filtering is a promising approach to eliminate redundancy so as reduce cost Previous efforts tailored effective solutions many applications, but left two essential questions unanswered: (1) theoretical filterability an inference workload guide application input techniques, thereby avoiding trial-and-error resource-constrained mobile applications; (2) xmlns:xlink="http://www.w3.org/1999/xlink">robust discriminability feature embedding allow be widely diverse tasks and content. To answer them, we first formulate problem theoretically compare hypothesis complexity models filters understand optimization potential. Then propose end-to-end learnable framework that covers most state-of-the-art methods surpasses them in embedding with robust discriminability. We design implement xmlns:xlink="http://www.w3.org/1999/xlink">InFi supports different modalities mobile-centric deployments. Comprehensive evaluations confirm our theoretical results show outperforms strong baselines applicability, accuracy, efficiency. can achieve 8.5× throughput save 95% bandwidth, while keeping over 90% video analytics on platforms.